Assignment 3 - Classification
Jacqueline Liu
You will need to load the core library for the course textbook:
library(ISLR)# Loading other libraries
library(tidyr)
library(stargazer)##
## Please cite as:## Hlavac, Marek (2018). stargazer: Well-Formatted Regression and Summary Statistics Tables.## R package version 5.2.2. https://CRAN.R-project.org/package=stargazerlibrary(kableExtra)
library(ggplot2)
library(dplyr)##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(GGally)##
## Attaching package: 'GGally'## The following object is masked from 'package:dplyr':
##
## nasalibrary(ggthemes)
library(ggExtra)
library(caret)## Loading required package: latticelibrary(rms)## Loading required package: Hmisc## Loading required package: survival##
## Attaching package: 'survival'## The following object is masked from 'package:caret':
##
## cluster## Loading required package: Formula##
## Attaching package: 'Hmisc'## The following objects are masked from 'package:dplyr':
##
## src, summarize## The following objects are masked from 'package:base':
##
## format.pval, units## Loading required package: SparseM##
## Attaching package: 'SparseM'## The following object is masked from 'package:base':
##
## backsolveExercise 3.1
In this problem, you will develop a model to predict whether a given car gets high or low gas mileage based on the Auto dataset from the ISLR package.
str(Auto)## 'data.frame': 392 obs. of 9 variables:
## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ...
## $ cylinders : num 8 8 8 8 8 8 8 8 8 8 ...
## $ displacement: num 307 350 318 304 302 429 454 440 455 390 ...
## $ horsepower : num 130 165 150 150 140 198 220 215 225 190 ...
## $ weight : num 3504 3693 3436 3433 3449 ...
## $ acceleration: num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ...
## $ year : num 70 70 70 70 70 70 70 70 70 70 ...
## $ origin : num 1 1 1 1 1 1 1 1 1 1 ...
## $ name : Factor w/ 304 levels "amc ambassador brougham",..: 49 36 231 14 161 141 54 223 241 2 ...summary(Auto)## mpg cylinders displacement horsepower
## Min. : 9.00 Min. :3.000 Min. : 68.0 Min. : 46.0
## 1st Qu.:17.00 1st Qu.:4.000 1st Qu.:105.0 1st Qu.: 75.0
## Median :22.75 Median :4.000 Median :151.0 Median : 93.5
## Mean :23.45 Mean :5.472 Mean :194.4 Mean :104.5
## 3rd Qu.:29.00 3rd Qu.:8.000 3rd Qu.:275.8 3rd Qu.:126.0
## Max. :46.60 Max. :8.000 Max. :455.0 Max. :230.0
##
## weight acceleration year origin
## Min. :1613 Min. : 8.00 Min. :70.00 Min. :1.000
## 1st Qu.:2225 1st Qu.:13.78 1st Qu.:73.00 1st Qu.:1.000
## Median :2804 Median :15.50 Median :76.00 Median :1.000
## Mean :2978 Mean :15.54 Mean :75.98 Mean :1.577
## 3rd Qu.:3615 3rd Qu.:17.02 3rd Qu.:79.00 3rd Qu.:2.000
## Max. :5140 Max. :24.80 Max. :82.00 Max. :3.000
##
## name
## amc matador : 5
## ford pinto : 5
## toyota corolla : 5
## amc gremlin : 4
## amc hornet : 4
## chevrolet chevette: 4
## (Other) :365Auto$originf <- factor(Auto$origin, labels = c("usa", "europe", "japan"))
with(Auto, table(originf, origin))## origin
## originf 1 2 3
## usa 245 0 0
## europe 0 68 0
## japan 0 0 79- Create a binary variable,
mpg01, that contains a 1 ifmpgcontains a value above its median, and a 0 ifmpgcontains a value below its median. You can compute the median using themedian()function. Note you may find it helpful to use thedata.frame()function to create a single data set containing bothmpg01and the otherAutovariables.
mpg01 <- ifelse(Auto$mpg > (median(Auto$mpg)), 1, 0)
Auto_new = data.frame(Auto, mpg01)
str(Auto_new)## 'data.frame': 392 obs. of 11 variables:
## $ mpg : num 18 15 18 16 17 15 14 14 14 15 ...
## $ cylinders : num 8 8 8 8 8 8 8 8 8 8 ...
## $ displacement: num 307 350 318 304 302 429 454 440 455 390 ...
## $ horsepower : num 130 165 150 150 140 198 220 215 225 190 ...
## $ weight : num 3504 3693 3436 3433 3449 ...
## $ acceleration: num 12 11.5 11 12 10.5 10 9 8.5 10 8.5 ...
## $ year : num 70 70 70 70 70 70 70 70 70 70 ...
## $ origin : num 1 1 1 1 1 1 1 1 1 1 ...
## $ name : Factor w/ 304 levels "amc ambassador brougham",..: 49 36 231 14 161 141 54 223 241 2 ...
## $ originf : Factor w/ 3 levels "usa","europe",..: 1 1 1 1 1 1 1 1 1 1 ...
## $ mpg01 : num 0 0 0 0 0 0 0 0 0 0 ...summary(Auto_new)## mpg cylinders displacement horsepower
## Min. : 9.00 Min. :3.000 Min. : 68.0 Min. : 46.0
## 1st Qu.:17.00 1st Qu.:4.000 1st Qu.:105.0 1st Qu.: 75.0
## Median :22.75 Median :4.000 Median :151.0 Median : 93.5
## Mean :23.45 Mean :5.472 Mean :194.4 Mean :104.5
## 3rd Qu.:29.00 3rd Qu.:8.000 3rd Qu.:275.8 3rd Qu.:126.0
## Max. :46.60 Max. :8.000 Max. :455.0 Max. :230.0
##
## weight acceleration year origin
## Min. :1613 Min. : 8.00 Min. :70.00 Min. :1.000
## 1st Qu.:2225 1st Qu.:13.78 1st Qu.:73.00 1st Qu.:1.000
## Median :2804 Median :15.50 Median :76.00 Median :1.000
## Mean :2978 Mean :15.54 Mean :75.98 Mean :1.577
## 3rd Qu.:3615 3rd Qu.:17.02 3rd Qu.:79.00 3rd Qu.:2.000
## Max. :5140 Max. :24.80 Max. :82.00 Max. :3.000
##
## name originf mpg01
## amc matador : 5 usa :245 Min. :0.0
## ford pinto : 5 europe: 68 1st Qu.:0.0
## toyota corolla : 5 japan : 79 Median :0.5
## amc gremlin : 4 Mean :0.5
## amc hornet : 4 3rd Qu.:1.0
## chevrolet chevette: 4 Max. :1.0
## (Other) :365- Explore the data graphically in order to investigate the association between
mpg01and the other features. Which of the other features seem most likely to be useful in predictingmpg01? Scatterplots and boxplots may be useful tools to answer this question. Describe your findings.
Scatterplot matrix seems to indicate that cylinders, dispacement, horsepower, weight, acceleration, year, and originf may be useful in predicting mpg01.
I first investigated the relationship of the variables to the continuous mpg variable. I then plotted several facet plots to examine how the relationship between mpg01 and other variables varies with the origin of the vehicle. Having checked the cross-tabulation of origin, it is clear that the data-set is slightly imbalanced and skewed towards cars from USA.
# Scatterplot matrix
Auto_new %>% select(-name) %>%
ggpairs(aes(color = originf, alpha = 0.2)) + theme_tufte(base_size = 8, base_family = "serif", ticks = TRUE)## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
# Boxplots
x <- Auto_new$cylinders
y <- Auto_new$mpg
boxplot(y ~ x, main = "", axes = FALSE, xlab = "", ylab = "",
pars = list(boxcol = "transparent", medlty = "blank", medpch=16, whisklty = c(1, 1),
medcex = 0.7, outcex = 0, staplelty = "blank"))
axis(1, at=1:length(unique(x)), label=sort(unique(x)), tick=F, family="serif")
axis(2, las=2, tick=F, family="serif")
text(min(x)/4, max(y)/1, pos = 4, family="serif",
"Boxplot of cylinders vs. miles per gallon (mpg)")
# Boxplot and scatterplot of horsepower vs. mpg
p <- ggplot(Auto_new, aes(horsepower, mpg)) + geom_point(alpha = 0.5) + theme_tufte(ticks = FALSE) + theme(axis.title=element_blank(), axis.text=element_blank())
p
ggMarginal(p, type = "boxplot", size = 10, fill="transparent")
# Scatterplot and density plot of acceleration vs. mpg
p1 <- ggplot(Auto_new, aes(acceleration, mpg)) + geom_point(alpha = 0.5) + theme_tufte(ticks=F) + theme(axis.title=element_blank(), axis.text=element_blank())
p1
ggMarginal(p1, type = "density", fill="transparent")
p3 <- ggplot(Auto_new, aes(mpg01, weight, color = factor(originf))) + geom_point(alpha = 0.5) + theme_tufte() + xlab("Miles per gallon of fuel (binary)") + ylab("Car weight") +
theme(axis.title.x = element_text(vjust=-0.5), axis.title.y = element_text(vjust=1))
p3 + facet_grid(cols = vars(Auto_new$originf))
p4 <- ggplot(Auto_new, aes(mpg, weight, color = factor(originf))) + geom_point(alpha = 0.5) + theme_tufte() + xlab("Miles per gallon of fuel") + ylab("Car weight") +
theme(axis.title.x = element_text(vjust=-0.5), axis.title.y = element_text(vjust=1))
p4 + facet_grid(cols = vars(Auto_new$originf))
p5 <- ggplot(Auto_new, aes(mpg01, acceleration, color = factor(originf))) + geom_point(alpha = 0.5) + theme_tufte() + xlab("Miles per gallon of fuel (binary)") + ylab("Acceleration") +
theme(axis.title.x = element_text(vjust=-0.5), axis.title.y = element_text(vjust=1))
p5 + facet_grid(cols = vars(Auto_new$originf))
- Split the data into a training set and a test set.
# The dataset is quite small with 392 observations. Randomly splitting the dataset 60/40:
set.seed(1)
# For replicating results
# Shuffle rows
rows <- sample(nrow(Auto_new))
# Randomly order data
Auto_new <- Auto_new[rows, ]
# Identify row to split on
split <- round(nrow(Auto_new)*0.60)
# Create train
train <- Auto_new[1:split, ]
# Create test
test <- Auto_new[(split+1):nrow(Auto_new), ]
# Check dimensions
dim(train)## [1] 235 11dim(test)## [1] 157 11- Perform logistic regression on the training data in order to predict
mpg01using the variables that seemed most associated withmpg01in (b). What is the test error of the model obtained?
The test error was 0.076 or 7.6%.
set.seed(1)
model1 <- glm(mpg01 ~ cylinders + displacement + horsepower + weight + originf, data = train, family = binomial)
summary(model1)##
## Call:
## glm(formula = mpg01 ~ cylinders + displacement + horsepower +
## weight + originf, family = binomial, data = train)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -2.4144 -0.1799 0.0791 0.3874 3.1176
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 11.581334 2.178981 5.315 1.07e-07 ***
## cylinders 0.041260 0.435128 0.095 0.92446
## displacement -0.008233 0.012010 -0.686 0.49301
## horsepower -0.053739 0.018114 -2.967 0.00301 **
## weight -0.001837 0.001000 -1.837 0.06620 .
## originfeurope 0.426551 0.735788 0.580 0.56210
## originfjapan 0.468557 0.772010 0.607 0.54390
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 325.77 on 234 degrees of freedom
## Residual deviance: 132.28 on 228 degrees of freedom
## AIC: 146.28
##
## Number of Fisher Scoring iterations: 7test.predict = predict(model1, test, type = "response")
test.predict.class = ifelse(test.predict > 0.5, 1, 0)
table(test$mpg01, test.predict.class, dnn = c("Actual Mileage", "Predicted Mileage"))## Predicted Mileage
## Actual Mileage 0 1
## 0 65 14
## 1 4 74mean(test.predict.class != test$mpg01)## [1] 0.1146497# Test error rate of 0.076 or 7.6%- Perform KNN on the training data, with several values of K, in order to predict
mpg01. Use only the variables that seemed most associated withmpg01in (b). What test errors do you obtain? Which value of K seems to perform the best on this data set?
Test errors for different values of K:
k = 3: 0.070 (7%)
k = 5: 0.082 (8.2%)
k = 20: 0.089 (8.9%)
k = 80: 0.096 (9.6%)
k = 3 seems to yield the lowest test error of 7%.
# Subset data set with variables most associated with `mpg01` in (b): cylinders, displacement, horsepower, weight, originf
head(Auto_new)Auto.subset <- Auto_new[,c(2,3,4,5,10,11)]
# Standardise variables for KNN
#Auto.knn <- scale(Auto.subset[,-5])
Auto.knn <- scale(Auto.subset[,-5])
# Set seed
set.seed(1)
# Divide into training and test data (60/40 split)
# Shuffle rows
rows <- sample(nrow(Auto.knn))
# Randomly order data
Auto.knn <- Auto.knn[rows, ]
# Identify row to split on
split <- round(nrow(Auto.knn)*0.60)
# Create train
train.k <- Auto.knn[1:split, ]
# Create test
test.k <- Auto.knn[(split+1):nrow(Auto.knn), ]
# Check dimensions
dim(train.k)## [1] 235 5dim(test.k)## [1] 157 5length(train.k)## [1] 1175# KNN with k = 5
library(class)
set.seed(1)
k1 <- knn(train.k[,-5], test.k[,-5], train.k[,5], k = 5)
mean(test.k[,5] != k1)## [1] 0.07006369# KNN with k = 20
library(class)
set.seed(1)
k1 <- knn(train.k[,-5], test.k[,-5], train.k[,5], k = 20)
mean(test.k[,5] != k1)## [1] 0.08280255# KNN with k = 80
library(class)
set.seed(1)
k1 <- knn(train.k[,-5], test.k[,-5], train.k[,5], k = 80)
mean(test.k[,5] != k1)## [1] 0.08917197# KNN with k = 3
library(class)
set.seed(1)
k1 <- knn(train.k[,-5], test.k[,-5], train.k[,5], k = 3)
mean(test.k[,5] != k1)## [1] 0.07643312Exercise 3.2
- Using a dataset of your choice develop a classification model. Provide a very basic description of what you are doing and why.
The dataset used is the “Boston” dataset from the MASS package in R. The dataset has 506 observations with 14 variables and records housing values in the suburbs of Boston. I would like to predict the per capita crime rate based on several socio-economic indicators. One such indicator could be housing values. Other indicators that could be considered are housing size, presence of industry (non-retail business), pupil-teacher ratio, etc. A classification model (e.g. logistic regression) could be run to predict the probability of the presence of low or high crime rate, which could be informative for allocation of law enforcement/ resources.
The variables in the dataset are:
crim: per capita crime rate by townzn: proportion of residential land zoned for lots over 25,000 sq. ft.indus: proportion of non-retail business acres per townchas: Charles River dummy variable (=1 if tract bounds river; 0 otherwise)nox: nitrogen oxides concentration (parts per 10 million)rm: average number of rooms per dwellingage: proportion of owner-occupied units built prior to 1940dis: weighted mean of distances to five Boston employment centresrad: index of accessibility to radial highwaystax: full-value property-tax rate per $10,000ptratio: pupil-teacher ratio by townblack: 1000(Bk - 0.63)^2 where Bk is the proportion of blacks by townlstat: lower status of the population (percent)medv: median value of owner-occupied homes in $1000s.
I re-classified some of the variables to prepare the dataset for analysis. I re-classified average room size by grouping them into 3 classes: “small” (if less than 4 rooms), “medium” (4 to 6 rooms), and “large” (more than or equal to 7 rooms). I also created a new binary variable for the response, crim (crim01).
library("MASS")##
## Attaching package: 'MASS'## The following object is masked from 'package:dplyr':
##
## selectdata("Boston")
str(Boston)## 'data.frame': 506 obs. of 14 variables:
## $ crim : num 0.00632 0.02731 0.02729 0.03237 0.06905 ...
## $ zn : num 18 0 0 0 0 0 12.5 12.5 12.5 12.5 ...
## $ indus : num 2.31 7.07 7.07 2.18 2.18 2.18 7.87 7.87 7.87 7.87 ...
## $ chas : int 0 0 0 0 0 0 0 0 0 0 ...
## $ nox : num 0.538 0.469 0.469 0.458 0.458 0.458 0.524 0.524 0.524 0.524 ...
## $ rm : num 6.58 6.42 7.18 7 7.15 ...
## $ age : num 65.2 78.9 61.1 45.8 54.2 58.7 66.6 96.1 100 85.9 ...
## $ dis : num 4.09 4.97 4.97 6.06 6.06 ...
## $ rad : int 1 2 2 3 3 3 5 5 5 5 ...
## $ tax : num 296 242 242 222 222 222 311 311 311 311 ...
## $ ptratio: num 15.3 17.8 17.8 18.7 18.7 18.7 15.2 15.2 15.2 15.2 ...
## $ black : num 397 397 393 395 397 ...
## $ lstat : num 4.98 9.14 4.03 2.94 5.33 ...
## $ medv : num 24 21.6 34.7 33.4 36.2 28.7 22.9 27.1 16.5 18.9 ...summary(Boston)## crim zn indus chas
## Min. : 0.00632 Min. : 0.00 Min. : 0.46 Min. :0.00000
## 1st Qu.: 0.08204 1st Qu.: 0.00 1st Qu.: 5.19 1st Qu.:0.00000
## Median : 0.25651 Median : 0.00 Median : 9.69 Median :0.00000
## Mean : 3.61352 Mean : 11.36 Mean :11.14 Mean :0.06917
## 3rd Qu.: 3.67708 3rd Qu.: 12.50 3rd Qu.:18.10 3rd Qu.:0.00000
## Max. :88.97620 Max. :100.00 Max. :27.74 Max. :1.00000
## nox rm age dis
## Min. :0.3850 Min. :3.561 Min. : 2.90 Min. : 1.130
## 1st Qu.:0.4490 1st Qu.:5.886 1st Qu.: 45.02 1st Qu.: 2.100
## Median :0.5380 Median :6.208 Median : 77.50 Median : 3.207
## Mean :0.5547 Mean :6.285 Mean : 68.57 Mean : 3.795
## 3rd Qu.:0.6240 3rd Qu.:6.623 3rd Qu.: 94.08 3rd Qu.: 5.188
## Max. :0.8710 Max. :8.780 Max. :100.00 Max. :12.127
## rad tax ptratio black
## Min. : 1.000 Min. :187.0 Min. :12.60 Min. : 0.32
## 1st Qu.: 4.000 1st Qu.:279.0 1st Qu.:17.40 1st Qu.:375.38
## Median : 5.000 Median :330.0 Median :19.05 Median :391.44
## Mean : 9.549 Mean :408.2 Mean :18.46 Mean :356.67
## 3rd Qu.:24.000 3rd Qu.:666.0 3rd Qu.:20.20 3rd Qu.:396.23
## Max. :24.000 Max. :711.0 Max. :22.00 Max. :396.90
## lstat medv
## Min. : 1.73 Min. : 5.00
## 1st Qu.: 6.95 1st Qu.:17.02
## Median :11.36 Median :21.20
## Mean :12.65 Mean :22.53
## 3rd Qu.:16.95 3rd Qu.:25.00
## Max. :37.97 Max. :50.00Boston$rmsize <- ifelse(Boston$rm < 4, 1,
ifelse(Boston$rm >= 4 & Boston$rm < 7, 2, 3))
table(Boston$rmsize)##
## 1 2 3
## 2 440 64Boston$rmsize <- factor(Boston$rmsize, labels = c("small", "medium", "large"))
summary(Boston$rmsize)## small medium large
## 2 440 64Boston %>%
ggpairs(aes(color = rmsize, alpha = 0.2)) + theme_tufte(base_size = 8, base_family = "serif", ticks = TRUE)## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
# Creating a binary variable for crime rate: crim01
# Assigning high crime rate (1) to crime rates equal and above the median
crim01 <- ifelse(Boston$crim >= (median(Boston$crim)), 1, 0)
Boston1 = data.frame(Boston, crim01)
str(Boston1)## 'data.frame': 506 obs. of 16 variables:
## $ crim : num 0.00632 0.02731 0.02729 0.03237 0.06905 ...
## $ zn : num 18 0 0 0 0 0 12.5 12.5 12.5 12.5 ...
## $ indus : num 2.31 7.07 7.07 2.18 2.18 2.18 7.87 7.87 7.87 7.87 ...
## $ chas : int 0 0 0 0 0 0 0 0 0 0 ...
## $ nox : num 0.538 0.469 0.469 0.458 0.458 0.458 0.524 0.524 0.524 0.524 ...
## $ rm : num 6.58 6.42 7.18 7 7.15 ...
## $ age : num 65.2 78.9 61.1 45.8 54.2 58.7 66.6 96.1 100 85.9 ...
## $ dis : num 4.09 4.97 4.97 6.06 6.06 ...
## $ rad : int 1 2 2 3 3 3 5 5 5 5 ...
## $ tax : num 296 242 242 222 222 222 311 311 311 311 ...
## $ ptratio: num 15.3 17.8 17.8 18.7 18.7 18.7 15.2 15.2 15.2 15.2 ...
## $ black : num 397 397 393 395 397 ...
## $ lstat : num 4.98 9.14 4.03 2.94 5.33 ...
## $ medv : num 24 21.6 34.7 33.4 36.2 28.7 22.9 27.1 16.5 18.9 ...
## $ rmsize : Factor w/ 3 levels "small","medium",..: 2 2 3 2 3 2 2 2 2 2 ...
## $ crim01 : num 0 0 0 0 0 0 0 0 0 0 ...summary(Boston1)## crim zn indus chas
## Min. : 0.00632 Min. : 0.00 Min. : 0.46 Min. :0.00000
## 1st Qu.: 0.08204 1st Qu.: 0.00 1st Qu.: 5.19 1st Qu.:0.00000
## Median : 0.25651 Median : 0.00 Median : 9.69 Median :0.00000
## Mean : 3.61352 Mean : 11.36 Mean :11.14 Mean :0.06917
## 3rd Qu.: 3.67708 3rd Qu.: 12.50 3rd Qu.:18.10 3rd Qu.:0.00000
## Max. :88.97620 Max. :100.00 Max. :27.74 Max. :1.00000
## nox rm age dis
## Min. :0.3850 Min. :3.561 Min. : 2.90 Min. : 1.130
## 1st Qu.:0.4490 1st Qu.:5.886 1st Qu.: 45.02 1st Qu.: 2.100
## Median :0.5380 Median :6.208 Median : 77.50 Median : 3.207
## Mean :0.5547 Mean :6.285 Mean : 68.57 Mean : 3.795
## 3rd Qu.:0.6240 3rd Qu.:6.623 3rd Qu.: 94.08 3rd Qu.: 5.188
## Max. :0.8710 Max. :8.780 Max. :100.00 Max. :12.127
## rad tax ptratio black
## Min. : 1.000 Min. :187.0 Min. :12.60 Min. : 0.32
## 1st Qu.: 4.000 1st Qu.:279.0 1st Qu.:17.40 1st Qu.:375.38
## Median : 5.000 Median :330.0 Median :19.05 Median :391.44
## Mean : 9.549 Mean :408.2 Mean :18.46 Mean :356.67
## 3rd Qu.:24.000 3rd Qu.:666.0 3rd Qu.:20.20 3rd Qu.:396.23
## Max. :24.000 Max. :711.0 Max. :22.00 Max. :396.90
## lstat medv rmsize crim01
## Min. : 1.73 Min. : 5.00 small : 2 Min. :0.0
## 1st Qu.: 6.95 1st Qu.:17.02 medium:440 1st Qu.:0.0
## Median :11.36 Median :21.20 large : 64 Median :0.5
## Mean :12.65 Mean :22.53 Mean :0.5
## 3rd Qu.:16.95 3rd Qu.:25.00 3rd Qu.:1.0
## Max. :37.97 Max. :50.00 Max. :1.0Boston1 %>%
ggpairs(aes(color = rmsize, alpha = 0.2)) + theme_tufte(base_size = 8, base_family = "serif", ticks = TRUE)## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero
## Warning in cor(x, y, method = method, use = use): the standard deviation is
## zero## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
plot1 <- ggplot(Boston1, aes(factor(crim01), medv, color = rmsize)) + geom_jitter(alpha = 0.5) + theme_tufte()
plot1
# Areas with a higher median value of owner-occupied homes seem to have lower crime ratesplot2 <- ggplot(Boston1, aes(factor(crim01), indus, color = rmsize)) + geom_boxplot(alpha = 0.5) + theme_tufte()
plot2 + facet_grid(.~ rmsize)
# Higher proportion of non-retail business in an area is associated with higher crime rates
plot3 <- ggplot(Boston1, aes(factor(crim01), nox, color = rmsize)) + geom_boxplot(alpha = 0.5) + theme_tufte()
plot3 + facet_grid(.~ rmsize)
# Higher NOx emissions in an area is also associated with higher crime rates
plot4 <- ggplot(Boston1, aes(factor(crim01), age, color = rmsize)) + geom_boxplot(alpha = 0.5) + theme_tufte()
plot4 + facet_grid(.~ rmsize)
# Newer houses seem to be associated with lower crime rates
plot5 <- ggplot(Boston1, aes(factor(crim01), tax, color = rmsize)) + geom_boxplot(alpha = 0.5) + theme_tufte()
plot5 + facet_grid(.~ rmsize)
# Higher levels of property tax rate seem to be associated with higher crime rates
plot6 <- ggplot(Boston1, aes(factor(crim01), ptratio, color = rmsize)) + geom_boxplot(alpha = 0.5) + theme_tufte()
plot6 + facet_grid(.~ rmsize)
# Lower pupil-teacher ratios may or may not be associated with crime rate levels
plot7 <- ggplot(Boston1, aes(factor(crim01), black, color = rmsize)) + geom_boxplot(alpha = 0.5) + theme_tufte()
plot7 + facet_grid(.~ rmsize)
# Proportion of black people in an area may or may not be associated with crime rate levels
plot8 <- ggplot(Boston1, aes(factor(crim01), lstat, color = rmsize)) + geom_boxplot(alpha = 0.5) + theme_tufte()
plot8 + facet_grid(.~ rmsize)
# Areas with higher percentage of low-income population could be associated with higher crime rates# Checking frequency distribution of key predictor: medv
ggplot(Boston1, aes(medv)) + geom_histogram() + theme_tufte()## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
- Implement the classifier (e.g. logistic regression) using implementations in user-written packages. Compare the results from the
glmfunction in base R to the results from your alternative implementation (e.g.lrmfunction inrmspackage).
The regression model using the glm function in base R yielded a lower test error rate of 17.8% while the model using the lrm function in the rms package yielded a slightly higher test error rate of 19.8%. However, the model using the lrm function yielded lower standard errors indicating that it may be a slightly more efficient model.
# Checking for class bias in response variable - Balanced
table(Boston1$crim01)##
## 0 1
## 253 253# Create test and training set
set.seed(1)
# For replicating results
# Shuffle rows
rows <- sample(nrow(Boston1))
# Randomly order data
Boston1 <- Boston1[rows, ]
# Identify row to split on
split <- round(nrow(Boston1)*0.60)
# Create train
bostontrain <- Boston1[1:split, ]
# Create test
bostontest <- Boston1[(split+1):nrow(Boston1), ]
# Check dimensions
dim(bostontrain)## [1] 304 16dim(bostontest)## [1] 202 16# Run logistic regression with `glm`
set.seed(1)
model3 <- glm(crim01 ~ medv + indus + nox + rmsize + age + lstat, data = bostontrain, family = binomial)
summary(model3)##
## Call:
## glm(formula = crim01 ~ medv + indus + nox + rmsize + age + lstat,
## family = binomial, data = bostontrain)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -2.31628 -0.28589 0.00324 0.20869 2.94632
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -8.48596 937.57164 -0.009 0.993
## medv -0.01673 0.04248 -0.394 0.694
## indus -0.06445 0.05037 -1.279 0.201
## nox 37.56499 6.97370 5.387 7.18e-08 ***
## rmsizemedium -11.45631 937.56385 -0.012 0.990
## rmsizelarge -10.24318 937.56413 -0.011 0.991
## age 0.01538 0.01163 1.322 0.186
## lstat -0.03583 0.05563 -0.644 0.519
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 421.10 on 303 degrees of freedom
## Residual deviance: 168.49 on 296 degrees of freedom
## AIC: 184.49
##
## Number of Fisher Scoring iterations: 14test.predict = predict(model3, bostontest, type = "response")
test.predict.class = ifelse(test.predict > 0.5, 1, 0)
table(bostontest$crim01, test.predict.class, dnn = c("Actual Crime Rate", "Predicted Crime Rate"))## Predicted Crime Rate
## Actual Crime Rate 0 1
## 0 88 18
## 1 18 78mean(test.predict.class != bostontest$crim01)## [1] 0.1782178# Run logistic regression with `lrm`
library(rms)
model4 <- lrm(crim01 ~ medv + indus + nox + rmsize + age + lstat, data = bostontrain)
print(model4, digits=4, strata.coefs = FALSE, coefs = TRUE)## Logistic Regression Model
##
## lrm(formula = crim01 ~ medv + indus + nox + rmsize + age + lstat,
## data = bostontrain)
##
## Model Likelihood Discrimination Rank Discrim.
## Ratio Test Indexes Indexes
## Obs 304 LR chi2 252.61 R2 0.753 C 0.950
## 0 147 d.f. 7 g 4.851 Dxy 0.900
## 1 157 Pr(> chi2) <0.0001 gr 127.807 gamma 0.901
## max |deriv| 0.06 gp 0.452 tau-a 0.451
## Brier 0.089
##
## Coef S.E. Wald Z Pr(>|Z|)
## Intercept -15.8313 39.4896 -0.40 0.6885
## medv -0.0167 0.0425 -0.39 0.6937
## indus -0.0644 0.0504 -1.28 0.2008
## nox 37.5650 6.9737 5.39 <0.0001
## rmsize=medium -4.1110 39.3043 -0.10 0.9167
## rmsize=large -2.8979 39.3109 -0.07 0.9412
## age 0.0154 0.0116 1.32 0.1860
## lstat -0.0358 0.0556 -0.64 0.5194
## lrmpredict = predict(model4, newdata = bostontest)
lrmpredict.class = ifelse(lrmpredict > 0.5, 1, 0)
table(bostontest$crim01, lrmpredict.class, dnn = c("Actual Crime Rate", "Predicted Crime Rate"))## Predicted Crime Rate
## Actual Crime Rate 0 1
## 0 91 15
## 1 25 71mean(lrmpredict.class != bostontest$crim01)## [1] 0.1980198- Use the
caretpackage to calculate the confusion matrix. Discuss the key results like sensitivity, specificity, kappa, etc. Calculate precision, recall, and F1 using any implementation on CRAN.
The model appears to have a relatively high accuracy rate of 0.82 (82%). The number of true positive cases (“Low crime rate”) was 88 (out of 106 cases), and number of true negative cases (“High crime rate”) was 78 (out of 96 cases). The prevalence (or prior probability) was 0.5248.
Sensitivity: Sensitivity was 0.8302. This means that 83% of samples with the event were predicted accurately.
Specificity: Specificity was 0.8125. This means that 81% of samples without the event were predicted accurately. The false-positive rate was then (1-0.8125) = 0.1875 or 18.8%.
Kappa: Kappa statistic was 0.6427, which is relatively close to 1. This means that there is some concordance between predicted and observed classes.
Precision: Precision was 0.8302 or 83%, which means that the model is quite precise.
Recall: Recall was 0.8302 or 83%, which is also a relatively high rate of recall.
F1: F1 was 0.8302 or 83%.
# Convert crim01 and test.predict.class to factors:
bostontest$crimtest <- factor(bostontest$crim01)
p_class <- factor(test.predict.class, levels = levels(bostontest[["crimtest"]])) # Create confusion matrix
confusionMatrix(p_class, bostontest$crimtest, mode = "everything")## Confusion Matrix and Statistics
##
## Reference
## Prediction 0 1
## 0 88 18
## 1 18 78
##
## Accuracy : 0.8218
## 95% CI : (0.7619, 0.8719)
## No Information Rate : 0.5248
## P-Value [Acc > NIR] : <2e-16
##
## Kappa : 0.6427
## Mcnemar's Test P-Value : 1
##
## Sensitivity : 0.8302
## Specificity : 0.8125
## Pos Pred Value : 0.8302
## Neg Pred Value : 0.8125
## Precision : 0.8302
## Recall : 0.8302
## F1 : 0.8302
## Prevalence : 0.5248
## Detection Rate : 0.4356
## Detection Prevalence : 0.5248
## Balanced Accuracy : 0.8213
##
## 'Positive' Class : 0
## # Precision : 0.8302
# Recall : 0.8302
# F1 : 0.8302- Produce the ROC curve and discuss the results.
The ROC curve appears to be plotted relatively far away from the baseline (the diagonal line). The Area Under the Curve (AUC) is 0.923, which is very close to 1, indicating that the model performs quite well.
# Store actual and predicted results in a dataframe
p <- data.frame(crim01 = bostontest$crim01)
p$results <- predict(model3, bostontest, type = "response")
#Plot ROC curve
library(pROC)## Type 'citation("pROC")' for a citation.##
## Attaching package: 'pROC'## The following objects are masked from 'package:stats':
##
## cov, smooth, varroc <- roc(p$crim01, p$results, levels = levels(as.factor(p$crim01)))
roc##
## Call:
## roc.default(response = p$crim01, predictor = p$results, levels = levels(as.factor(p$crim01)))
##
## Data: p$results in 106 controls (p$crim01 0) < 96 cases (p$crim01 1).
## Area under the curve: 0.9213# Area under the curve: 0.9213# Plot ROC curve
plot(roc, legacy.axes = TRUE)
- Assess class imbalance and implement any of the strategies to mitigate it.
If there is severe class imbalance, we should expect the ROC curve and the lift curve to be very similar. To assess if there is class imbalance, I plotted a lift curve and found that it was very dissimilar to the ROC curve, indicating that classes are already quite balanced. However, the lift plot also indicated that the model may actually not be informative.
One way to mitigate the presence of class imbalance is to alter the cut-off/ threshold, which could affect the predictive power of the model. Since the threshold was initially set at p > 0.5, I used different threshold levels to assess the effect on performance metrics including the test error rate, sensitivity and specificity.
A lower threshold of p > 0.3 improved the sensitivity of the model but decreased specificity. The test error rate was also higher at 20%. While a slightly higher threshold using Youden’s J index lowered the sensitivity of the model but achieved very high specificity. The test error rate was then lower at 15.8%. This means that with a lower threshold, the model predicts events more accurately. A higher threshold on the other hand allows the model to predict non-events more accurately.
# Assess class imbalance
# Plot lift curve
p$crim01 <- as.factor(p$crim01)
labs <- c(results = "Logistic Regression")
lift <- lift(x = crim01 ~ results, data = p)
lift##
## Call:
## lift.formula(x = crim01 ~ results, data = p)
##
## Models: results
## Event: 0 (52.5%)#plot(roc, legacy.axes = TRUE)
xyplot(lift, auto.key = list(columns = 2, lines = TRUE, points= FALSE))
# The lift curve is very different from the ROC curve, which could indicate that classes are quite balanced. # Using a different threshold
# Confusion matrix for initial threshold of p > 0.5
test.predict = predict(model3, bostontest, type = "response")
test.predict.class = ifelse(test.predict > 0.5, 1, 0)
tab1 <- table(bostontest$crim01, test.predict.class, dnn = c("Actual Crime Rate", "Predicted Crime Rate"))
# Confusion matrix for intial threshold of p > 0.9
new.predict = predict(model3, bostontest, type = "response")
new.predict.class = ifelse(new.predict > 0.3, 1, 0)
tab2 <- table(bostontest$crim01, new.predict.class, dnn = c("Actual Crime Rate", "Predicted Crime Rate"))
# Confusion matrix for threshold determined by pROC package
thresh <- coords(roc, x = "best", best.method = "youden")
new.predict.class1 = ifelse(new.predict > thresh, 1, 0)## Warning in new.predict > thresh: longer object length is not a multiple of
## shorter object lengthtab3 <- table(bostontest$crim01, new.predict.class1, dnn = c("Actual Crime Rate", "Predicted Crime Rate"))
tab1.e <- mean(test.predict.class != bostontest$crim01)
tab2.e <- mean(new.predict.class != bostontest$crim01)
tab3.e <- mean(new.predict.class1 != bostontest$crim01)
sen1 <- sensitivity(tab1)
sen2 <- sensitivity(tab2)
sen3 <- sensitivity(tab3)
spec1 <- specificity(tab1)
spec2 <- specificity(tab2)
spec3 <- specificity(tab3)
error.rate <- tibble(Threshold = c("p > 0.5", "p > 0.3", "Youden's J index"), Error_Rate = c(tab1.e, tab2.e, tab3.e), Sensitivity = c(sen1, sen2, sen3), Specificity = c(spec1, spec2, spec3))
error.rate- Implement a naive Bayes classifier. Briefly describe why it may be useful here and highlight it’s basic properties. Show the implementation and discuss the choice of parameters.
The naive Bayes model is a conditional probability model, which simplifies probabilities of the predictor values by assuming that all of the predictors are independent of the others. Naive Bayes classifiers may be used because they can be trained efficiently and perform competitively. Another advantage of naive Bayes classifiers is that it can be used on a small number of training data to estimate probabilities. Since the dataset is not very large, I applied the naive Bayes model on a sample that is resampled through cross-validation (10-fold). The Kappa here is 64% (which is similar to that in the logistic regression model) and accuracy is also comparable to that in the logistic regression model. Overall, the performance of the naive Bayes classifier is comparable to that of the logistic regression.
# Create naive Bayes classifier
library(caret)
library(e1071)##
## Attaching package: 'e1071'## The following object is masked from 'package:Hmisc':
##
## imputelibrary(klaR)
# using naiveBayes() function
x = Boston1[,-16]
x = x[,-1]
y = Boston1$crim01
yf = as.factor(y)
nbmodel = train(x, yf, method = 'nb', trControl = trainControl(method = 'cv', number = 10))## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 51## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 52## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 51## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 52## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 51## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 51## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 51## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 51## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 51## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 51## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 51## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 1## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 2## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 3## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 4## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 5## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 6## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 7## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 8## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 9## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 10## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 11## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 12## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 13## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 14## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 15## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 16## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 17## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 18## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 19## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 20## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 21## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 22## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 23## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 24## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 25## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 26## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 27## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 28## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 29## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 30## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 31## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 32## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 33## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 34## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 35## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 36## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 37## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 38## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 39## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 40## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 41## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 42## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 43## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 44## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 45## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 46## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 47## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 48## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 49## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 50## Warning in FUN(X[[i]], ...): Numerical 0 probability for all classes with
## observation 51nbmodel## Naive Bayes
##
## 506 samples
## 14 predictor
## 2 classes: '0', '1'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 454, 456, 456, 455, 456, 455, ...
## Resampling results across tuning parameters:
##
## usekernel Accuracy Kappa
## FALSE 0.8241418 0.6480698
## TRUE 0.8599864 0.7197323
##
## Tuning parameter 'fL' was held constant at a value of 0
## Tuning
## parameter 'adjust' was held constant at a value of 1
## Accuracy was used to select the optimal model using the largest value.
## The final values used for the model were fL = 0, usekernel = TRUE
## and adjust = 1.